<!DOCTYPE html>
<html lang="zh-CN">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>QST (Quantized Side Tuning) 完整流程深度解析</title>
    <script src="https://cdn.tailwindcss.com"></script>
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/styles/atom-one-dark.min.css">
    <script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.9.0/highlight.min.js"></script>
    <script src="https://unpkg.com/lucide@latest"></script>
    <style>
        :root {
            --bg-primary: #111827;
            --bg-secondary: #1f2937;
            --border-color: #374151;
            --text-primary: #d1d5db;
            --text-secondary: #9ca3af;
            --text-heading: #ffffff;
            --accent-light: #38bdf8;
            --accent-dark: #818cf8;
            --green-light: #34d399;
            --green-dark: #10b981;
        }
        html {
            scroll-behavior: smooth;
        }
        body {
            font-family: 'Inter', -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji";
            background-color: var(--bg-primary);
            color: var(--text-primary);
        }
        .step-card {
            background-color: var(--bg-secondary);
            border: 1px solid var(--border-color);
            border-radius: 1rem;
            padding: 2rem;
            margin-bottom: 2.5rem;
            box-shadow: 0 10px 15px -3px rgba(0, 0, 0, 0.1), 0 4px 6px -2px rgba(0, 0, 0, 0.05);
            transition: transform 0.3s ease, box-shadow 0.3s ease;
        }
        .step-card:hover {
            transform: translateY(-5px);
            box-shadow: 0 20px 25px -5px rgba(31, 41, 55, 0.5), 0 10px 10px -5px rgba(31, 41, 55, 0.4);
        }
        h2 {
            font-size: 1.875rem;
            font-weight: 700;
            color: var(--text-heading);
            margin-bottom: 1.5rem;
            display: flex;
            align-items: center;
        }
        h3 {
            font-size: 1.5rem;
            font-weight: 600;
            color: #e5e7eb;
            margin-bottom: 1rem;
            margin-top: 1.5rem;
        }
        .icon {
            margin-right: 0.75rem;
            color: var(--accent-light);
        }
        pre code.hljs {
            border-radius: 0.5rem;
            padding: 1rem;
            font-size: 0.875rem;
            background-color: #0d1117;
        }
        .diagram {
            background-color: var(--bg-primary);
            border: 1px solid var(--border-color);
            padding: 1.5rem;
            border-radius: 0.75rem;
            margin-top: 1.5rem;
        }
        .gradient-text {
            background: linear-gradient(to right, var(--accent-light), var(--accent-dark));
            -webkit-background-clip: text;
            -webkit-text-fill-color: transparent;
        }
        .step-connector {
            display: flex;
            justify-content: center;
            align-items: center;
            margin: -2rem 0 0.5rem 0;
            color: #4b5563;
        }
    </style>
</head>
<body>

    <div class="container mx-auto px-4 py-12">
        <!-- Header -->
        <header class="text-center mb-16">
            <h1 class="text-4xl md:text-5xl font-bold tracking-tight text-white">
                <span class="gradient-text">QST (Quantized Side Tuning)</span>
            </h1>
            <h1 class="text-4xl md:text-5xl font-bold tracking-tight text-white mt-2">完整流程深度解析</h1>
            <p class="mt-6 text-lg text-gray-400 max-w-3xl mx-auto">
                一个结合了模型量化与侧调优的高效微调技术的可视化指南，助您深入理解从理论到实践的全过程。
            </p>
        </header>

        <!-- 1. Core Workflow Overview -->
        <section class="step-card" id="overview">
            <h2>
                <i data-lucide="git-merge" class="icon"></i>
                1. 核心流程架构
            </h2>
            <p class="text-gray-400 mb-8">
                QST 的执行流程以 `qst.py` 中的 `train()` 函数为总指挥中心，协调调用各个子模块完成任务。这个非线性的架构图展示了各部分之间的调用关系和逻辑流，点击模块可跳转至对应详解。
            </p>
            <div class="diagram">
                 <svg width="100%" viewBox="0 0 800 550" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
                    <style>
                        .center-box { fill: #3730a3; stroke: #818cf8; }
                        .main-box { fill: #1e40af; stroke: #38bdf8; cursor: pointer; transition: all 0.2s ease-in-out; }
                        .main-box:hover { fill: #1d4ed8; transform: scale(1.02); }
                        .sub-box { fill: #1f2937; stroke: #4b5563; cursor: pointer; transition: all 0.2s ease-in-out; }
                        .sub-box:hover { background: #374151; transform: scale(1.02); }
                        .text-lg { font-family: Inter, sans-serif; font-size: 18px; font-weight: bold; fill: white; text-anchor: middle; pointer-events: none; }
                        .text-md { font-family: Inter, sans-serif; font-size: 14px; fill: #e5e7eb; text-anchor: middle; pointer-events: none; }
                        .text-sm { font-family: Inter, sans-serif; font-size: 12px; fill: #9ca3af; text-anchor: middle; pointer-events: none; }
                        .flow-line { stroke: #4b5563; stroke-width: 2; marker-end: url(#arrow); }
                        .sub-flow-line { stroke: #374151; stroke-width: 1.5; marker-end: url(#arrow-sm); }
                    </style>
                    <defs>
                        <marker id="arrow" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse"><path d="M 0 0 L 10 5 L 0 10 z" fill="#4b5563" /></marker>
                        <marker id="arrow-sm" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="5" markerHeight="5" orient="auto-start-reverse"><path d="M 0 0 L 10 5 L 0 10 z" fill="#374151" /></marker>
                    </defs>

                    <!-- Center Box -->
                    <rect x="300" y="225" width="200" height="100" rx="10" class="center-box"/>
                    <text x="400" y="265" class="text-lg">train() 函数</text>
                    <text x="400" y="290" class="text-sm">(`qst.py` 总指挥)</text>

                    <!-- Main Modules -->
                    <a xlink:href="#quantization">
                        <rect x="50" y="80" width="220" height="100" rx="10" class="main-box"/>
                        <text x="160" y="120" class="text-md">get_accelerate_model()</text>
                        <text x="160" y="145" class="text-sm">模型加载、量化与构建</text>
                    </a>
                    <line class="flow-line" x1="310" y1="245" x2="220" y2="180" />
                    
                    <a xlink:href="#training">
                         <rect x="530" y="80" width="220" height="100" rx="10" class="main-box"/>
                         <text x="640" y="120" class="text-md">Seq2SeqTrainer</text>
                         <text x="640" y="145" class="text-sm">训练、评估与保存</text>
                    </a>
                    <line class="flow-line" x1="490" y1="245" x2="580" y2="180" />

                    <a xlink:href="#data-deep-dive">
                        <rect x="290" y="420" width="220" height="100" rx="10" class="main-box"/>
                        <text x="400" y="460" class="text-md">make_data_module()</text>
                        <text x="400" y="485" class="text-sm">数据集加载与预处理</text>
                    </a>
                    <line class="flow-line" x1="400" y1="325" x2="400" y2="420" />

                    <!-- Sub Modules for get_accelerate_model -->
                    <a xlink:href="#quantization">
                        <rect x="20" y="0" width="160" height="60" rx="8" class="sub-box"/>
                        <text x="100" y="33" class="text-sm">2. 模型加载与量化</text>
                    </a>
                    <line class="sub-flow-line" x1="100" y1="60" x2="100" y2="80" />
                    
                    <a xlink:href="#side-tuning">
                        <rect x="190" y="0" width="160" height="60" rx="8" class="sub-box"/>
                        <text x="270" y="33" class="text-sm">3. 侧调优网络构建</text>
                    </a>
                     <line class="sub-flow-line" x1="270" y1="60" x2="210" y2="80" />
                    
                    <a xlink:href="#config-deep-dive">
                         <rect x="10" y="200" width="180" height="60" rx="8" class="sub-box"/>
                        <text x="100" y="233" class="text-sm">4. 配置与核心模块</text>
                    </a>
                     <line class="sub-flow-line" x1="100" y1="180" x2="100" y2="200" />

                    <a xlink:href="#model-deep-dive">
                        <rect x="200" y="200" width="180" height="60" rx="8" class="sub-box"/>
                        <text x="290" y="233" class="text-sm">5. 模型组装与前向传播</text>
                    </a>
                    <line class="sub-flow-line" x1="290" y1="200" x2="330" y2="225" />

                     <!-- Sub Modules for Trainer -->
                    <a xlink:href="#training">
                        <rect x="550" y="0" width="180" height="60" rx="8" class="sub-box"/>
                        <text x="640" y="33" class="text-sm">7. 训练循环与内存效率</text>
                    </a>
                     <line class="sub-flow-line" x1="640" y1="60" x2="640" y2="80" />
                 </svg>
            </div>
        </section>

        <div class="step-connector">
            <i data-lucide="arrow-down" class="w-8 h-8"></i>
        </div>

        <!-- 2. Model Loading & 4-bit Quantization -->
        <section class="step-card" id="quantization">
            <h2>
                <i data-lucide="gem" class="icon"></i>
                2. 模型加载与 4-bit 量化
            </h2>
            <div class="grid md:grid-cols-2 gap-8 items-start">
                <div>
                    <h3>理论阐述</h3>
                    <p class="text-gray-400">
                        这是 QST 的第一阶段。目标是通过量化技术大幅压缩基础 LLM 的权重，从而降低静态内存占用。权重从标准的 16-bit 浮点数转换为 4-bit 格式（如 NF4），每个参数的存储空间减少了 75%。
                    </p>
                    <div class="diagram mt-6">
                        <svg width="100%" viewBox="0 0 400 150" xmlns="http://www.w3.org/2000/svg">
                            <defs>
                                <linearGradient id="grad1" x1="0%" y1="0%" x2="100%" y2="0%">
                                    <stop offset="0%" style="stop-color:rgb(56,189,248);stop-opacity:1" />
                                    <stop offset="100%" style="stop-color:rgb(129,140,248);stop-opacity:1" />
                                </linearGradient>
                            </defs>
                            <!-- Original Weights -->
                            <rect x="10" y="10" width="140" height="130" rx="10" fill="#1f2937" stroke="#38bdf8" stroke-width="2"/>
                            <text x="80" y="40" font-family="Inter" font-size="14" fill="#e5e7eb" text-anchor="middle" font-weight="bold">原始权重 (BF16)</text>
                            <text x="80" y="70" font-family="monospace" font-size="10" fill="#9ca3af" text-anchor="middle">[0.123, -1.456, ...]</text>
                            <text x="80" y="85" font-family="monospace" font-size="10" fill="#9ca3af" text-anchor="middle">[2.789, 0.005, ...]</text>
                            <text x="80" y="115" font-family="Inter" font-size="12" fill="#d1d5db" text-anchor="middle">70B 模型 ≈ 140GB</text>

                            <!-- Arrow -->
                            <path d="M 160 75 L 220 75" stroke="url(#grad1)" stroke-width="3" marker-end="url(#arrowhead-quant)"/>
                            <defs>
                                <marker id="arrowhead-quant" markerWidth="10" markerHeight="7" refX="0" refY="3.5" orient="auto">
                                    <polygon points="0 0, 10 3.5, 0 7" fill="url(#grad1)"/>
                                </marker>
                            </defs>
                            <text x="190" y="65" font-family="Inter" font-size="10" fill="#9ca3af" text-anchor="middle">4-bit量化</text>
                            
                            <!-- Quantized Weights -->
                            <rect x="230" y="10" width="140" height="130" rx="10" fill="#1f2937" stroke="#34d399" stroke-width="2"/>
                            <text x="300" y="40" font-family="Inter" font-size="14" fill="#e5e7eb" text-anchor="middle" font-weight="bold">量化权重 (NF4)</text>
                            <text x="300" y="70" font-family="monospace" font-size="10" fill="#9ca3af" text-anchor="middle">[5, 13, ...]</text>
                            <text x="300" y="85" font-family="monospace" font-size="10" fill="#9ca3af" text-anchor="middle">[1, 8, ...]</text>
                            <text x="300" y="115" font-family="Inter" font-size="12" fill="#d1d5db" text-anchor="middle">70B 模型 ≈ 35GB</text>
                        </svg>
                        <p class="text-center text-sm mt-2 text-gray-400">通过 `bitsandbytes` 库实现高效量化</p>
                    </div>
                </div>
                <div>
                    <h3>源码实现</h3>
                    <p class="text-gray-400 mb-4">
                        此过程在 `get_accelerate_model` 函数中通过 `BitsAndBytesConfig` 实现。
                    </p>
                    <pre><code class="language-python hljs"># qst.py: get_accelerate_model() 函数片段

# 1. 定义量化配置
compute_dtype = (torch.float16 if args.fp16 else (torch.bfloat16 if args.bf16 else torch.float32))
model = AutoModelForCausalLM.from_pretrained(
    args.model_name_or_path,
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=args.bits == 4,
        bnb_4bit_quant_type=args.quant_type,
        bnb_4bit_use_double_quant=args.double_quant,
        bnb_4bit_compute_dtype=compute_dtype,
    ),
    torch_dtype=(torch.float32 if args.fp16 else (torch.bfloat16 if args.bf16 else torch.float32)),
)

# 2. 为 k-bit 训练准备模型
if not args.full_finetune:
    model = prepare_model_for_kbit_training(
        model, 
        use_gradient_checkpointing=args.gradient_checkpointing
    )
                    </code></pre>
                </div>
            </div>
        </section>

        <div class="step-connector">
             <i data-lucide="arrow-down" class="w-8 h-8"></i>
        </div>

        <!-- 3. Side Tuning Network Construction (RESTORED) -->
        <section class="step-card" id="side-tuning">
            <h2>
                <i data-lucide="network" class="icon"></i>
                3. 侧调优 (Side Tuning) 网络构建
            </h2>
            <div class="grid md:grid-cols-2 gap-8 items-start">
                <div>
                    <h3>理论阐述</h3>
                    <p class="text-gray-400">
                        这是 QST 的核心创新。在冻结的、量化的 LLM 旁边，为每一层创建一个小型的“侧模块”。梯度只在这些轻量模块中流动，完全绕过庞大的基础 LLM，从而避免存储海量的中间激活值。
                    </p>
                    <div class="diagram mt-6">
                        <svg width="100%" viewBox="0 0 400 300" xmlns="http://www.w3.org/2000/svg">
                            <style>
                                .frozen-box { fill: #374151; stroke: #6b7280; }
                                .trainable-box { fill: #064e3b; stroke: #34d399; }
                                .adapter-box { fill: #1e3a8a; stroke: #38bdf8; }
                                .text-main { font-family: Inter, sans-serif; font-size: 12px; fill: #e5e7eb; text-anchor: middle; }
                                .text-frozen { font-family: Inter, sans-serif; font-size: 10px; fill: #9ca3af; text-anchor: middle; }
                                .text-trainable { font-family: Inter, sans-serif; font-size: 10px; fill: #a7f3d0; text-anchor: middle; }
                                .data-flow { stroke: #9ca3af; stroke-width: 1.5; fill: none; marker-end: url(#data-arrow-side); }
                                .gradient-flow { stroke: url(#grad-side); stroke-width: 2; fill: none; marker-end: url(#grad-arrow-side); }
                            </style>
                            <defs>
                                <marker id="data-arrow-side" viewBox="0 0 10 10" refX="5" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse"><path d="M 0 0 L 10 5 L 0 10 z" fill="#9ca3af" /></marker>
                                <marker id="grad-arrow-side" viewBox="0 0 10 10" refX="5" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse"><path d="M 0 0 L 10 5 L 0 10 z" fill="url(#grad-side)" /></marker>
                                <linearGradient id="grad-side" x1="0%" y1="0%" x2="100%" y2="0%">
                                    <stop offset="0%" stop-color="#34d399" />
                                    <stop offset="100%" stop-color="#38bdf8" />
                                </linearGradient>
                            </defs>
                            
                            <!-- Main LLM Layer -->
                            <rect x="20" y="100" width="120" height="80" rx="8" class="frozen-box"/>
                            <text x="80" y="135" class="text-main">LLM Layer i</text>
                            <text x="80" y="155" class="text-frozen">(冻结, 4-bit)</text>

                            <!-- Side Network -->
                            <rect x="240" y="20" width="120" height="40" rx="8" class="adapter-box"/>
                            <text x="300" y="45" class="text-main">Downsample</text>
                            <rect x="240" y="100" width="120" height="80" rx="8" class="trainable-box"/>
                            <text x="300" y="135" class="text-main">Side Module i</text>
                            <text x="300" y="155" class="text-trainable">(可训练, BF16)</text>
                            <rect x="240" y="220" width="120" height="40" rx="8" class="adapter-box"/>
                            <text x="300" y="245" class="text-main">Upsample</text>

                             <!-- Data flow paths -->
                            <path class="data-flow" d="M 80 100 Q 80 80, 100 80 L 140 80"/> <!-- Input to LLM -->
                            <path class="data-flow" d="M 80 180 Q 80 200, 100 200 L 140 200"/> <!-- Output from LLM -->
                            <path class="gradient-flow" d="M 140 140 L 240 140" />
                            <text x="190" y="130" class="text-frozen" font-size="9">捕获隐藏状态</text>
                            <path class="gradient-flow" d="M 280 100 V 60" />
                            <path class="gradient-flow" d="M 280 180 V 220" />

                        </svg>
                         <p class="text-center text-sm mt-2 text-gray-400">QST 第 i 层架构图：梯度只在右侧的侧网络中流动</p>
                    </div>
                </div>
                <div>
                    <h3>源码实现</h3>
                    <p class="text-gray-400 mb-4">
                        通过自定义的 `QSTConfig` 和模型包装类 (`QSTLlamaForCausalLM` 或 `QSTOPTForCausalLM`) 来实现。
                    </p>
                    <pre><code class="language-python hljs"># qst.py: get_accelerate_model() 函数片段

if not args.full_finetune:
    config = AutoConfig.from_pretrained(args.model_name_or_path)
    
    # 1. 定义侧网络的配置
    qst_config = QSTConfig(...)

    # 2. 将基础模型包装成 QST 模型
    if 'Llama' in args.model_name_or_path:
        model = QSTLlamaForCausalLM(model, config, qst_config)
    elif 'opt' in args.model_name_or_path:
        model = QSTOPTForCausalLM(model, config, qst_config)
    
    # 3. 设置各模块的数据类型
    for name, module in model.named_modules():
        if 'qst' in name or 'downsample' in name:
            if args.bf16:
                module = module.to(torch.bfloat16)
        if 'norm' in name:
            module = module.to(torch.float32)
                    </code></pre>
                </div>
            </div>
        </section>


        <div class="step-connector">
            <i data-lucide="arrow-down" class="w-8 h-8"></i>
       </div>

        <!-- 4. Deep Dive into Config & Modules (Renumbered) -->
        <section class="step-card" id="config-deep-dive">
            <h2>
                <i data-lucide="cogs" class="icon"></i>
                4. 深入配置与模块: `QSTConfig` 与 `AdapterLinear`
            </h2>
            <div class="grid md:grid-cols-2 gap-8 items-start">
                <div>
                    <h3>理论阐述</h3>
                    <p class="text-gray-400">
                        侧网络的具体行为和结构由 `QSTConfig` 定义，而其核心构建块则是 `AdapterLinear` 模块。理解这两个组件是掌握 QST 实现的关键。
                    </p>
                    <h4 class="text-lg font-semibold text-gray-200 mt-6 mb-2">QSTConfig: 侧网络的蓝图</h4>
                    <p class="text-gray-400">
                        这个配置类决定了侧网络的“胖瘦”和“行为”。关键参数如 `r` 控制了降维的程度，直接影响侧网络的大小和可训练参数量。
                    </p>
                    <h4 class="text-lg font-semibold text-gray-200 mt-6 mb-2">AdapterLinear: 高效的信息桥梁</h4>
                    <p class="text-gray-400">
                        `AdapterLinear` 模块是 `downsample` 操作的核心。它通过一个“降维 &rarr; 激活 &rarr; 升维”的瓶颈结构，以极少的参数量，从主干网络的高维隐藏状态中提取出对侧网络有用的信息。
                    </p>
                    <div class="diagram mt-6">
                         <svg width="100%" viewBox="0 0 400 100" xmlns="http://www.w3.org/2000/svg">
                            <style>.text-label{font-family: Inter, sans-serif; font-size: 10px; fill: #9ca3af; text-anchor: middle;}.text-dim{font-family: monospace; font-size: 11px; fill: #e5e7eb; text-anchor: middle;}</style>
                            <defs><marker id="arrow-cfg" viewBox="0 0 10 10" refX="8" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse"><path d="M 0 0 L 10 5 L 0 10 z" fill="#818cf8"></path></marker></defs>
                            <!-- Input -->
                            <text x="30" y="20" class="text-label">输入</text>
                            <text x="30" y="35" class="text-dim">in_features</text>
                            
                            <!-- Adapter A -->
                            <path d="M 60 50 L 110 50" fill="none" stroke="#818cf8" stroke-width="2" marker-end="url(#arrow-cfg)"></path>
                            <rect x="110" y="35" width="70" height="30" rx="5" fill="#1e3a8a" stroke="#38bdf8"></rect>
                            <text x="145" y="54" class="text-label">Adapter A</text>
                            
                            <!-- Bottleneck -->
                            <text x="215" y="20" class="text-label">瓶颈</text>
                            <text x="215" y="35" class="text-dim">(dim: r)</text>
                            
                            <!-- Activation + Adapter B -->
                            <path d="M 180 50 L 250 50" fill="none" stroke="#818cf8" stroke-width="2" marker-end="url(#arrow-cfg)"></path>
                            <rect x="250" y="35" width="100" height="30" rx="5" fill="#064e3b" stroke="#34d399"></rect>
                            <text x="300" y="54" class="text-label">激活 + Adapter B</text>

                            <!-- Output -->
                            <path d="M 350 50 L 400 50" fill="none" stroke="#818cf8" stroke-width="2" marker-end="url(#arrow-cfg)"></path>
                            <text x="400" y="20" class="text-label">输出</text>
                            <text x="400" y="35" class="text-dim">out_features</text>
                        </svg>
                         <p class="text-center text-sm mt-2 text-gray-400">AdapterLinear 模块内部结构</p>
                    </div>
                </div>
                <div>
                    <h3>源码实现 (`QSTConfig.py`)</h3>
                    <pre><code class="language-python hljs">@dataclass
class QSTConfig(PretrainedConfig):
    """
    这是一个用于QST方法的配置类，定义了所有与
    QST侧网络相关的超参数。
    """
    add_layer_norm_before_adapter: bool = False
    add_layer_norm_after_adapter: bool = True
    activation: str = "swish"
    r: int = 16  # 降维/升维的中间维度大小
    alpha_r: int = 16  # 用于缩放的alpha值
    dropout: float = 0.1
    fan_in_fan_out: bool = False
    peft_hidden_size: int = 16


class AdapterLinear(nn.Module):
    """
    QST中的核心构建块，由一个降维和一个升维线性层组成。
    """
    def __init__(self, in_features, out_features, r, ...):
        super().__init__()
        # 降维层：将输入从 in_features 映射到 r
        self.adapter_A = nn.Linear(in_features, r, bias=False)
        # 升维层：将中间维度 r 映射回 out_features
        self.adapter_B = nn.Linear(r, out_features, bias=False)
        self.activation = Activations(activation.lower())
        # ... 其他初始化 ...

    def forward(self, x):
        # 核心计算：降维 -> 激活 -> 升维
        x = self.adapter_A(x)
        x = self.activation(x)
        y = self.adapter_B(x)
        return y
                    </code></pre>
                </div>
            </div>
        </section>

        <div class="step-connector">
            <i data-lucide="arrow-down" class="w-8 h-8"></i>
       </div>

        <!-- 5. Deep Dive into the Model (Renumbered) -->
        <section class="step-card" id="model-deep-dive">
            <h2>
                <i data-lucide="code-2" class="icon"></i>
                5. 深入模型: `QSTLlamaModel` 组装与前向传播
            </h2>
            <div class="grid md:grid-cols-2 gap-8 items-start">
                <div>
                    <h3>模型组装 (`__init__`)</h3>
                    <p class="text-gray-400">
                        `QSTLlamaModel` 初始化时，将量化后的原模型层设为 `backbone`，并根据 `QSTConfig` 为其并行创建三个核心可训练组件。
                    </p>
                    <ul class="list-disc list-inside text-gray-400 mt-4 space-y-2">
                        <li><b class="text-green-400">`downsample`</b>: `AdapterLinear` 模块，用于将 `backbone` 的高维输出降维。</li>
                        <li><b class="text-yellow-400">`z`</b>: 可学习的门控参数，动态平衡新旧信息。</li>
                        <li><b class="text-blue-400">`qst_layers`</b>: 轻量的侧网络层，执行学习任务。</li>
                    </ul>
                     <pre><code class="language-python hljs mt-4"># modeling_llama_qst.py: QSTLlamaModel.__init__()

# 主干网络：这是原始的、被冻结的LLM层
self.backbone = llm.layers

# 降采样模块：将主干网络的输出维度降低
self.downsample = nn.ModuleList([...])

# 参数z：一个可学习的门控参数
self.z = nn.ParameterList([...])

# QST层：这就是“侧网络”，更小、更轻量
self.qst_layers = nn.ModuleList([...])
                    </code></pre>
                </div>
                <div>
                    <h3>前向传播 (`forward`)</h3>
                    <p class="text-gray-400 mb-4">
                        `forward` 方法定义了信息如何在主干网络和侧网络之间流动，这是 QST 区别于其他方法的精髓所在。
                    </p>
                    <div class="diagram">
                        <svg width="100%" viewBox="0 0 400 250" xmlns="http://www.w3.org/2000/svg">
                             <style>
                                .box { rx: 5; ry: 5; }
                                .backbone-box { fill: #374151; stroke: #6b7280; }
                                .side-box { fill: #064e3b; stroke: #34d399; }
                                .op-box { fill: #1e3a8a; stroke: #38bdf8; }
                                .flow-line { fill: none; stroke: #9ca3af; stroke-width: 1.5; marker-end: url(#data-arrow-model); }
                                .label-text { font-family: Inter, sans-serif; font-size: 10px; fill: #d1d5db; text-anchor: middle; }
                                .formula-text { font-family: monospace; font-size: 11px; fill: #e5e7eb; }
                            </style>
                             <defs>
                                <marker id="data-arrow-model" viewBox="0 0 10 10" refX="5" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse"><path d="M 0 0 L 10 5 L 0 10 z" fill="#9ca3af" /></marker>
                            </defs>
                            <!-- Layer i-1 -->
                            <text x="50" y="20" class="label-text">From Layer i-1</text>
                            <path class="flow-line" d="M 120 40 L 120 70" />
                            <path class="flow-line" d="M 280 40 L 280 125" />

                            <!-- Backbone Path -->
                            <rect x="70" y="70" width="100" height="40" class="box backbone-box"/>
                            <text x="120" y="95" class="label-text">Backbone[i]</text>
                            <path class="flow-line" d="M 120 110 L 120 140" />
                            <text x="120" y="125" class="label-text">h_i</text>
                            
                            <!-- Downsample -->
                            <rect x="70" y="140" width="100" height="30" class="box op-box"/>
                            <text x="120" y="160" class="label-text">Downsample(h_i)</text>

                            <!-- Gating -->
                            <circle cx="200" cy="155" r="15" class="box op-box" />
                            <text x="200" y="158" class="label-text" font-size="14" fill="white">+</text>
                            <path class="flow-line" d="M 170 155 L 185 155" />
                            <path class="flow-line" d="M 215 155 L 230 155" />
                            
                            <!-- Side Path -->
                            <path class="flow-line" d="M 280 125 A 80 30 0 0 0 200 155" />
                            
                            <!-- z weights -->
                            <text x="150" y="145" class="formula-text">(1-z)</text>
                            <text x="240" y="130" class="formula-text">z</text>

                            <!-- Side Layer -->
                            <rect x="230" y="140" width="100" height="40" class="box side-box" />
                            <text x="280" y="165" class="label-text">Side Layer[i]</text>

                            <!-- Output -->
                            <path class="flow-line" d="M 120 170 L 120 200" />
                            <path class="flow-line" d="M 280 180 L 280 200" />
                            <text x="120" y="215" class="label-text">To Backbone[i+1]</text>
                            <text x="280" y="215" class="label-text">To Side Layer[i+1]</text>
                        </svg>
                        <p class="text-center text-sm mt-2 text-gray-400">`forward` 方法核心逻辑：通过门控 `z` 融合主干与侧网络信息</p>
                    </div>
                </div>
            </div>
        </section>

        <div class="step-connector">
             <i data-lucide="arrow-down" class="w-8 h-8"></i>
        </div>
        
        <!-- 6. Data Loading & Preprocessing (NEW) -->
        <section class="step-card" id="data-deep-dive">
            <h2>
                <i data-lucide="database-zap" class="icon"></i>
                6. 数据集加载与预处理
            </h2>
            <div class="grid md:grid-cols-2 gap-8 items-start">
                <div>
                    <h3>理论阐述</h3>
                    <p class="text-gray-400">
                        `make_data_module` 函数是数据处理的核心。它负责根据用户指定的 `dataset` 参数，从 Hugging Face Hub 或本地加载数据，然后通过一系列转换，将其统一为包含 `input` 和 `output` 字段的标准格式。
                    </p>
                     <h4 class="text-lg font-semibold text-gray-200 mt-6 mb-2">数据格式化</h4>
                     <p class="text-gray-400">
                        针对 Alpaca、Chip2 等不同格式的数据集，函数内部有专门的逻辑将其转换为统一的指令微调格式。例如，对于 Alpaca 数据集，它会根据有无输入，选择不同的提示词模板。
                    </p>
                    <h4 class="text-lg font-semibold text-gray-200 mt-6 mb-2">数据打包 (`DataCollator`)</h4>
                    <p class="text-gray-400">
                        `DataCollatorForCausalLM` 负责将格式化后的样本打包成批次 (batch)。它会进行分词 (tokenization)、截断、填充 (padding)，并构建用于因果语言模型训练的 `labels`，其中 `input` 部分的 `labels` 会被设置为 `-100` 以在计算损失时被忽略。
                    </p>
                </div>
                <div>
                    <h3>源码实现 (`qst.py`)</h3>
                    <pre><code class="language-python hljs"># qst.py: make_data_module() 函数片段

def make_data_module(tokenizer: transformers.PreTrainedTokenizer, args) -> Dict:
    # ...
    def format_dataset(dataset, dataset_format):
        # 以为 Alpaca 数据集为例
        if (dataset_format == 'alpaca'):
            dataset = dataset.map(extract_alpaca_dataset, remove_columns=['instruction'])
        # ... 其他数据集格式化逻辑 ...
        return dataset

    # 加载和格式化数据
    dataset = load_data(args.dataset)
    dataset = format_dataset(dataset, args.dataset_format)

    # ... 省略了数据集分割和采样逻辑 ...

    data_collator = DataCollatorForCausalLM(
        tokenizer=tokenizer,
        source_max_len=args.source_max_len,
        target_max_len=args.target_max_len,
        train_on_source=args.train_on_source,
        predict_with_generate=args.predict_with_generate,
    )
    return dict(train_dataset=train_dataset, data_collator=data_collator)

# qst.py: DataCollatorForCausalLM 类
@dataclass
class DataCollatorForCausalLM(object):
    # ...
    def __call__(self, instances: Sequence[Dict]) -> Dict[str, torch.Tensor]:
        # ... Tokenize sources and targets ...

        for tokenized_source, tokenized_target in zip(...):
            input_ids.append(torch.tensor(tokenized_source + tokenized_target))
            # 核心：将 source 部分的 label 设为 IGNORE_INDEX (-100)
            labels.append(
                torch.tensor(
                    [IGNORE_INDEX] * len(tokenized_source) + copy.deepcopy(tokenized_target)
                )
            )
        
        # ... 对 input_ids 和 labels 进行填充 ...
        return dict(input_ids=input_ids, labels=labels, ...)

                    </code></pre>
                </div>
            </div>
        </section>


        <div class="step-connector">
             <i data-lucide="arrow-down" class="w-8 h-8"></i>
        </div>

        <!-- 7. Training & Weight Saving (Renumbered) -->
        <section class="step-card" id="training">
             <h2>
                <i data-lucide="bar-chart-3" class="icon"></i>
                7. 训练与内存效率
            </h2>
            <div class="grid md:grid-cols-2 gap-8 items-start">
                <div>
                    <h3>理论阐述</h3>
                    <p class="text-gray-400">
                        训练时，只有侧网络的参数是可训练的。这导致优化器状态极小。保存时，也只需保存这部分轻量的、包含所有任务知识的权重，而非整个几十GB的基础模型。
                    </p>
                    <div class="diagram mt-6">
                        <svg width="100%" viewBox="0 0 400 200" xmlns="http://www.w3.org/2000/svg">
                           <style>
                                .bar-weights { fill: #3b82f6; }
                                .bar-optimizer { fill: #8b5cf6; }
                                .bar-activations { fill: #ec4899; }
                                .axis-text { font-family: Inter, sans-serif; font-size: 10px; fill: #9ca3af; }
                                .title-text { font-family: Inter, sans-serif; font-size: 12px; fill: #e5e7eb; text-anchor: middle; font-weight: bold;}
                           </style>
                           <!-- Axes -->
                           <line x1="30" y1="170" x2="380" y2="170" stroke="#4b5563" stroke-width="1"/>
                           <line x1="30" y1="20" x2="30" y2="170" stroke="#4b5563" stroke-width="1"/>
                           <text x="15" y="25" class="axis-text" transform="rotate(-90, 15, 95)">内存占用</text>
                           
                           <!-- Full Finetune Bars -->
                           <text x="110" y="185" class="title-text">全量微调</text>
                           <rect x="80" y="130" width="60" height="40" class="bar-weights"/>
                           <rect x="80" y="50" width="60" height="80" class="bar-optimizer"/>
                           <rect x="80" y="20" width="60" height="30" class="bar-activations"/>

                           <!-- QST Bars -->
                           <text x="290" y="185" class="title-text">QST</text>
                           <rect x="260" y="160" width="60" height="10" class="bar-weights"/>
                           <rect x="260" y="155" width="60" height="5" class="bar-optimizer"/>
                           <rect x="260" y="145" width="60" height="10" class="bar-activations"/>

                            <!-- Legend -->
                           <rect x="340" y="30" width="10" height="10" class="bar-weights" />
                           <text x="355" y="38" class="axis-text">权重</text>
                           <rect x="340" y="50" width="10" height="10" class="bar-optimizer" />
                           <text x="355" y="58" class="axis-text">优化器</text>
                           <rect x="340" y="70" width="10" height="10" class="bar-activations" />
                           <text x="355" y="78" class="axis-text">激活值</text>
                        </svg>
                        <p class="text-center text-sm mt-2 text-gray-400">QST 在权重、优化器和激活值三方面都显著降低了内存占用</p>
                    </div>
                </div>
                <div>
                    <h3>源码实现</h3>
                     <p class="text-gray-400 mb-4">
                        通过 `print_trainable_parameters` 验证参数量，并通过 `SavePeftModelCallback` 实现智能保存。
                    </p>
                    <pre><code class="language-python hljs"># qst.py: print_trainable_parameters() 函数
def print_trainable_parameters(args, model):
    trainable_params = 0
    all_param = 0
    for name, param in model.named_parameters():
        # 冻结基础模型的参数
        if "backbone" in name or "model.layers" in name:
            param.requires_grad = False
        all_param += param.numel()
        if param.requires_grad:
            trainable_params += param.numel()
    
    print(f"trainable: {100 * trainable_params / all_param:.4f}%")
    # -> 输出一个极小的比例，例如 0.2%

# qst.py: SavePeftModelCallback 类
class SavePeftModelCallback(transformers.TrainerCallback):
    def save_model(self, args, state, kwargs):
        print('Saving QST checkpoint...')
        checkpoint_folder = os.path.join(args.output_dir, f"checkpoint-{state.global_step}")
        
        # 核心：调用模型自定义的保存方法
        # 只保存可训练的侧网络权重
        kwargs["model"].save_qst_state(checkpoint_folder)

    def on_save(self, args, state, control, **kwargs):
        self.save_model(args, state, kwargs)
        return control
                    </code></pre>
                </div>
            </div>
        </section>

    </div>

    <footer class="text-center py-8 border-t border-gray-800">
        <p class="text-gray-500">可视化页面由 Gemini 生成</p>
    </footer>

    <script>
        hljs.highlightAll();
        lucide.createIcons();
    </script>
</body>
</html>

